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Background/Context: 


Enrollment in eighth-grade algebra doubled from the 1990s to 2012 (National Center for 
Education Statistics, 2013) as research demonstrated that Algebra I operates as a “gateway” to 
advanced mathematics courses in high school (Adelman, 2006), and that students who succeed in 
Algebra I in middle school have more success in mathematics throughout high school. However, 
prior work was heavily prone to selection bias, focusing on high-performing students. Other 
work examines outcomes of taking eighth-grade algebra for students with different levels of 
readiness and highlights negative effects for average or lower performing students (e.g., reduced 
test scores [Loveless, 2008], decreased likelihood of passing geometry by Grade 11 and Algebra 
II by Grade 12 [Clotfelter, Ladd, & Vigdor, 2013]). However, these studies have important 
limitations; they focus on district(s) within one state and/or only examine short-term outcomes, 
and no study examines graduation outcomes among average and lower-performing students. 


Purpose/Objective/Research Question: 


Given rapid expansion of access to eighth-grade algebra, the need to examine short- and longer- 
term outcomes for a// students is urgent. To address this gap, we conducted an exploration study 
funded by a grant from the Institute of Education Sciences (IES) to examine the following 
research questions in two distinct districts: 

1. What is the “impact” of placement into eighth-grade algebra on short-term outcomes 
(attendance, math course performance) across the transition to high school, and on 
longer-term high school milestones (completing math course requirements, graduation)? 

2. To what extent do “impacts” of placement into eighth-grade algebra vary for students 
with different levels of prior readiness? 


Setting: 

To broaden generalizability of our results, we examined data from two large and diverse 
educational settings—Elk Grove Unified School District (EGUSD) in California, and Chicago 
Public Schools (CPS). Tables B-2 and B-3 display student demographics for the districts. 


Population/Participants/Subjects: 


We examined secondary, historical data from six cohorts of students from EGUSD (22,429 
students, graduating classes of 2011-2016), and four cohorts of students from CPS (100,628 
students, graduating classes of 2013-2016). All EGUSD middle schools and approximately one 
third of CPS K-8 schools offered eighth-grade algebra. 


Research Design: 


To address selection bias, this quasi-experimental study employs a propensity weighting design, 
where students within a district not enrolled in eighth-grade algebra are weighted to more closely 
resemble students enrolled in eighth-grade algebra—maximizing internal validity in the absence 
of random assignment. 


Analysis: 


We calculated propensity weights using a selection model for each cohort, generating a predicted 
value for each student’s likelihood of enrolling in eighth-grade algebra given his or her 
background characteristics and prior achievement before eighth grade (see Tables B-2 and B-3 
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for all selection model variables with and without weights). Because not all schools in Chicago 
offered eighth-grade algebra, we calculated propensity weights using the sample of students 
attending schools that offered eighth-grade algebra. The estimates from that model were applied 
to all students to calculate propensity weights, and only students in schools that did not offer the 
course were included in the comparison group. 


We used the resulting predicted values to calculate the average treatment effect on the treated 
(ATT). Nearly all variables from the selection model had standardized mean differences of less 
than d=0.25 post-weighting. When applicable, we combined propensity weights with inverse 
probability weights (IPW) to account for missingness in outcomes. We generated levels of 
readiness to take eighth-grade algebra based on students’ seventh-grade math test scores, and 
final course grade in seventh-grade mathematics, and then interacted each readiness level with 
whether they took eighth-grade algebra (see Table B-4 for definitions of readiness). We used a 
regression to estimate the effects of taking eighth-grade algebra for students with different levels 
of readiness, controlling for student characteristics, school, and cohort, weighted by the product 
of the ATT and IPW for each outcome.’ Analyses were run separately for each district. 


Findings/Results: 


Short-term outcomes. In both sites, few students with very low readiness (lowest grades and test 
scores) were placed into eighth-grade algebra. With regard to attendance in eighth grade, 
students who took algebra missed less instructional time than students who did not take algebra, 
though this effect was not significant in Chicago. With regard to course performance, placement 
in eighth-grade algebra resulted in small decreases in math GPA in eighth grade, potentially as a 
result of more difficult coursework, and decreases were larger for students with lower prior 
grades and for students with average prior test scores (middle of the distribution of performance 
ratings) in both sites. Math course grades recovered by the end of ninth grade, and were higher 
for students in Chicago who were placed in eighth-grade algebra relative to their counterparts; 
positive effects on ninth-grade math GPA were larger for students with average prior test scores. 


Longer-term outcomes. All students who took algebra in eighth grade were more likely to 
complete Algebra I, Geometry, and Algebra II before eleventh grade, leaving time for 
Precalculus and Calculus, and effects on completion of requirements were larger for students 
with higher prior grades and test scores in both sites. Placement into eighth-grade algebra had no 
impact on high school graduation in EGUSD, but had a positive impact in Chicago, and this 
effect was larger for students with average or above-average prior grades and test scores. Results 
for each site are presented in Figures 1-14 and Tables B-5 and B-6. 


Conclusions: 


Though preliminary, our study suggests the following conclusions: 
= Although placement in eighth-grade algebra may initially result in lower math course 
performance in eighth-grade, we observed consistently positive, and no negative impacts 
for students in high school. 
= Positive impacts on course performance and graduation were sometimes larger for 
students who did not attain the highest prior math course grades or test scores, and 


' Chicago analyses exclude school fixed-effects because treatment status and school effects are collinear. 
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positive impacts on completion of math course requirements before eleventh grade were 
larger for higher performing students. 

" Relatively few students with the lowest math course grades or test scores in seventh grade 
took algebra in eighth grade, limiting generalizability for these students. 
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Appendix B. Tables and Figures 
Figures 1-7. EGUSD results. 


Figure 1. Percent of instructional time missed in eighth grade (interactions with prior 
grades and test scores not significant). 
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t=p<.1,*=p<.05,**=p<.01, *** =p <.001. Shown here are significance levels for pairwise comparisons of 
students who took eighth-grade algebra and those that did not. Significance values are not indicated if the interaction 
was not significant. 

Note: Grades refers to student seventh grade math GPA average, while Scores 1-5 refer to the student’s performance 
level on the California seventh grade math test. Presented here are weighted means (propensity weight and IPW) for 
each group. 


Figure 2. Eighth-grade math GPA (interactions with prior grades and test scores 
significant). 
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Figure 3. Ninth-grade math GPA (interaction with prior grades marginally significant, 
interaction with prior test scores significant). 
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Figure 4. Whether a student repeated Algebra I (interactions with prior grades and test 
scores significant). 
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Figure 5. Whether a student took Algebra II (interaction with prior grades marginally 
significant, interaction with prior test scores significant). 
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Figure 6. Whether a student completed core math requirements before eleventh grade 
(interactions with prior grades and test scores significant). 
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Figure 7. Whether a student graduated in four years (interaction with prior test scores 
marginally significant). 
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Figures 8-14. CPS results. 


Figure 8. Number of days missed school in 8th grade (interactions with prior grades and 
test scores significant) 
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*=p<.05, ** =p<.01, *** =p <.001. Shown here are significance levels for pairwise comparisons of students 
who took eighth-grade algebra and those that did not. Significance values are not indicated if the interaction was not 
significant. 

Note: Grades refers to student seventh grade math GPA average, while Stanines 1-9 refer to the student’s 
performance level on the ISAT seventh grade math test. 
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Figure 9. Eighth-grade math GPA (interactions with prior grades and test scores 
significant) 
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Figure 10. Ninth-grade math GPA (interaction with prior test scores significant) 
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*=p<.05,** =p<.01, *** =p<.001. 


Figure 11. Whether a student repeated Algebra (interactions with prior grades and test 
scores significant) 
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Figure 12. Whether a student took Algebra II (interaction with prior grades significant) 
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Figure 13. Whether a student completed core math requirements before elventh grade 
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Figure 14. Whether a student graduated in four years (interactions with prior grades and 
test scores significant) 


Overalls | *** 


Gace AI 
Grace 6, *** 
Grace C | es +++ 

Grade D | 

Grace F ees 


Stanine 9 
Stanine 8 
Stanine 7 ee, *** 
Stanine 6 a + 
Stanine 5 es + + 
Stanine 4 es | ++ 
Stanine 3 
Senin? I _ 
Sinire1 Se 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


m= No Algebra in 8th Grade mw Agebra in 8th Grade 


*=p<.05,** =p<.01, *** =p<.001. 


SREE Spring 2018 Conference Abstract 


14 


Table B-1. List and description of outcomes 


Outcome Description 

Absences in 8" grade Continuous; Percent of instructional time missed in 8" grade (EGUSD); number of days missed in 
eighth grade (CPS) 

Math GPA in 8" grade Continuous; End-of-course GPA of all math classes taken in gn grade (if more than one) 

Math GPA in 9" grade Continuous; End-of-course GPA of all math classes taken in gh grade (if more than one) 

Repeated Algebra I Binary; Did the student repeat Algebra I at least once? 

Took Algebra II Binary; Did the student take Algebra II? 

Completed core math before | Binary; Did the student complete (pass all sections of) Algebra I, Geometry, and Algebra II before 

11" grade 11" grade (leaving room to take Pre-calculus and Calculus in high school)? 


Graduated in 4 years 


Binary; Did the student graduate from high school within four years? 


Table B-2. Site demographics and selection model/final analysis variables, with and without weights (EGUSD) 


EGUSD 
All Students | Students who Students who Standardized Mean Differences 
took Algebra did not take 
in 8" grade | Algebra in 8™ 
grade 
Not weighted Weighted 
Number of students 22,449 7,463 14,986 
Selection Model Factors 
Ethnicity 
African American students 18% 11% 22% -0.28 -0.06 
American Indian 1% <1% 1% -0.02 0.01 
Asian 19% 27% 15% 0.31 -0.05 
Filipino 5% 7% 4% 0.13 -0.04 
Latino students 25% 18% 29% -0.24 -0.02 
Pacific Islander 2% 2% 2% 0.01 -0.01 
White 25% 30% 23% 0.16 0.13 
Multi-ethnic 4% 3% 4% -0.05 0.03 
Parent education level 
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Graduate degree 11% 16% 8% 0.26 0.16 
College degree 22% 29% 19% 0.25 0.04 
Some college 28% 25% 29% -0.10 -0.05 
High school diploma 24% 18% 27% -0.23 -0.05 
Did not graduate high school 10% 8% 11% -0.10 -0.08 
Declined to state 5% 4% 5% -0.04 0.00 
Male students 50% 45% 53% -0.15 0.00 
IEP status in 7" grade* 8% 1% 11% -0.37 -0.01 
LEP status in 7” grade* 13% 10% 20% -0.28 -0.09 
Free/reduced-priced lunch* 56% 42% 59% -0.35 -0.14 
Mean days suspended in Te grade <l <1 <1 -0.35 0.00 
Mean % of days absent in 7 grade 4% 2% 4% -0.45 -0.01 
Mean 7" grade GPA* 2.69 3.43 Doe 1.36 0.11 
Mean 7" grade mathematics test score 350 406 332 1.80 0.317* 
(scaled)* 
Mean 7" grade math course GPA 2.42 Be 1.97 1.42 0.11 
Readiness Factors 
(added to final analysis model) 
EGUSD 7" grade math test scores 
Far Below Basic | 5% <1% 8% 
Below Basic | 17% 1% 25% 
Basic | 30% 9% 41% 
Proficient | 32% 49% 24% 
Advanced | 16% 42% 2% 
7" grade math course performance 
F | 12% 1% 18% 
C-D | 47% 21% 61% 
A-B | 40% 78% 21% 
*Also included in the selection and final analysis model are missingness indicators of these binary status variables 
** Although this is not below the 0.25 threshold, it greatly improved from its unweighted SMD of 1.80. It, and all variables listed, are included in the final 
analysis models to account for additional variance that weighting does not correct. 
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Table B-3. Site demographics and selection model/final analysis variables, with and without weights (CPS) 


CPS 
All Students Standardized Mean 
Students | who took Differences 
Algebra in : (comparing students 
th Students who did not take 
8" grade algebra in 8" grade who took Algebra to 
those who did not in 
schools that did not 
offer Algebra) 
In Schools In ere ae F Weighted 
Overall offering ae an wee 
Algebra Algebra 
Number of students 100,628 13,982 86,700 28,018 58,682 --- --- 
Selection Model Factors 
Ethnicity 
African American students 42% 25% 45% 24% 55% -0.70 -0.56- 
Asian 3% 8% 3% 4% 2% 0.26 0.21 
Latino students 45% 49% 44% 61% 36% 0.40 0.35" 
White 9% 16% 7% 9% 7% 0.22 0.14 
Other 1% 1% 1% 1% 1% 0.08 0.06 
Male students 51% 46% 51% 53% 51% -0.01 0.02 
IEP status in 7" grade 13% 3% 15% 17% 14% -0.08 0.03 
LEP status in 7” grade 7% 2% 8% 13% 6% 0.00 0.06 
Free/reduced-priced lunch 84% 74% 86% 86% 86% -0.16 -0.15 
Mean days suspended in 7" grade 0.74 0.22 0.82 0.56 0.94 -0.24 -0.11 
Mean % of days absent in 7" grade 95% 97% 95% 95% 95% 0.35 0.16 
Mean 7" math grade GPA 221. 3.14 2.05 1.96 2.01 0.58 0.06 
Mean 7" grade mathematics test score 254 283 248 247 249 0.86 0.21 
Mean 7" grade reading test score 236 260 232 231 233 0.61 0.14 
Readiness Factors 
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(added to final analysis model) 
CPs 7® grade math test scores (CPS 
stanines) 
1 (mean math test scores = 202) 5% 0% 6% 5% 6% 
2 (mean math test scores = 215) 7% 0% 8% 8% 8% 
3 (mean math test scores = 227) 12% 1% 14% 14% 14% 
4 (mean math test scores = 239) 17% 4% 19% 21% 18% 
5 (mean math test scores = 253) 20% 11% 22% 25% 20% 
6 (mean math test scores = 267) 17% 22% 16% 17% 16% 
7 (mean math test scores = 281) 12% 26% 9% 8% 10% 
8 (mean math test scores = 299) 7% 23% 5% 3% 6% 
9 (mean math test scores = 328) 3% 12% 1% 1% 2% 
7" grade math course performance 
F 8% 1% 9% 10% 8% 
D 17% 4% 20% 20% 19% 
C 34% 17% 37% 39% 36% 
B 26% 35% 24% 24% 24% 
A 15% 44% 10% 7% 12% 


Note: Test scores for the ISAT tests taken in Illinois are grouped in four categories: Academic Warning (4% students in our sample), Below Standards (22% 
students in our sample), Meets Standards (52% students in our sample) and Exceeds Standards (17% in our sample). The vast majority of students taking algebra 
in eighth grade were either meeting or exceeding standards. For Chicago, we created a more fine division of the test scores to better understand the effects of 
taking algebra among students with different levels of readiness defined by their test scores. 


Table B-4. Description of readiness factors 


Readiness level 3: Average grade of F 


Outcome Description in EGUSD Description in CPS 
7th grade math Readiness level 1: Average grade of A or B Readiness level 1: Average grade of A 
GPA Readiness level 2: Average grade of C or D Readiness level 2: Average grade of B 


Readiness level 3: Average grade of C 
Readiness level 4: Average grade of D 
Readiness level 5: Average grade of F 


7th grade 
scores 


math test | Readiness level 1: Advanced proficiency score on 
California Standards Test (CST) 

Readiness level 2: Proficient proficiency score on CST 
Readiness level 3: Basic proficiency score on CST 


Readiness level 1: First stanine on Illinois 
Standards Achievement Test (ISAT) 
Readiness level 2: Second stanine on ISAT 
Readiness level 3: Third stanine on ISAT 
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CST 


Readiness level 4: Below basic proficiency score on CST 
Readiness level 5: Far below basic proficiency score on 


Readiness level 4: Fourth stanine on ISAT 
Readiness level 5: Fifth stanine on ISAT 
Readiness level 6: Sixth stanine on ISAT 
Readiness level 7: Seventh stanine on ISAT 
Readiness level 8: Eighth stanine on ISAT 
Readiness level 9: Ninth stanine on ISAT 


Table B-5. Coefficients, standard errors, and significance for each outcome (EGUSD). 


Transition from 8" to 9" grade 


High School Achievement & Graduation 


Math GPA | Math GPA | Absences Repeated Took Completed | Graduated 
in 8" Grade | in 9" Grade | in 8" grade | Algebra I Algebra II | core math in 4 years 
before 11" 
grade 
Test Readiness 
Algebra p=.001 p= .004 p= .06 p<.001 p<.001 p<.001 p=.001 
No Algebra in 8th | (omitted) 
Algebra in 8th | -0.33 0.95 -0.03 0.59 0.49 -0.14 0.22 
(0.65) (0.51) (0.01) (0.12) (0.12) (0.13) (0.09) 
Test Readiness p=.31 p=.01 p=.59 p= .04 p=. p= .0l p= .56 
Far Below Basic | (omitted) 
Below Basic | 0.09 0.23 -0.01 -0.06 0.17 -0.10 0.04 
(0.06) (0.09) (.00) (.05) (0.04) (0.03) (0.04) 
Basic | 0.17 0.37 -0.01 -0.07 0.24 -0.38 0.04 
0.13 (0.14) (.01) (0.13) (0.08) (0.12) (0.06) 
Proficient | 0.18 0.58 -0.01 0.01 0.30 -0.62 0.02 
(0.14) (0.16) (0.01) (0.14) (0.08) (0.12) (0.07) 
Advanced | 0.14 0.75 -0.01 0.00 0.30 -.68 0.01 
(0.15) (0.19) (.01) (0.14) (0.08) (0.13) (0.07) 
Algebra x Test Readiness p<.001 p=.001 p= ds p<.001 p<.001 p<.001 p= .06 
Algebra & Below Basic | -0.12 -0.38 0.03 -0.27 -0.32 0.30 -0.11 
(0.66) (0.53) (0.02) (0.13) (0.13) (0.15) (0.10) 
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Algebra & Basic | -0.26 -0.86 0.03 -0.36 -0.39 0.51 -0.19 
(0.65) (0.51) (0.01) (0.12) (0.12) (0.14) (0.10) 
Algebra & Proficient | -0.18 -0.94 0.02 -0.46 -0.44 0.78 -0.21 
(0.65) (0.51) (0.01) (0.12) (0.12) (0.14) (0.09) 
Algebra & Advanced | 0.02 -0.95 0.02 -0.47 -0.45 0.86 -0.21 
(0.65) (0.52) (0.01) (0.12) (0.12) (0.14) (0.10) 
GPA Readiness 
Algebra p<.001 p= .02 p<.001 p<.001 p<.001 p<.001 p=.58 
No Algebra in 8th | (omitted) 
Algebra in 8th | -0.42 -0.001 -0.003 0.02 0.18* 0.07 0.03 
(0.02) (0.01) (0.00) (0.09) (0.08) (0.04) (0.07) 
GPA Readiness p= .09 p=.91 p= .52 p= .02 p<.001 p<.001 p= .08 
F | (omitted) 
C-D | -0.03 -0.04 0.002 -0.31 O20F2% -0.12 0.12" 
(0.04) (0.03) (0.00) (0.08) (0.06) (0.06) (0.6) 
A-B | -0.39 -0.07 0.01 -0.26 0347" -0.43*** 0.10 
(0.09) (0.13) (0.01) (0.08) (0.07) (0.07) (0.06) 
Algebra x GPA Readiness p<.001 p=.07 p=.13 p<.001 p=.08 p<.001 p=.69 
Algebra & C-D | -0.08 0.01 -0.001 0,22” -0.10 2054 -0.03 
(0.04) (.02) (.00) (0.09) (0.08) (0.04) (0.08) 
Algebra & A-B | 0.50 -0.07 -0.01 0.08 -0.14 0.65*** -0.01 
(0.12) (0.24) (0.01) (0.09) (0.08) (0.04) (0.07) 


Note: Shown here are the coefficients for each effect in the model and the standard error (in parentheses). P-values represent significance values for the main 
effects and interactions. Pairwise significance comparing algebra and non-algebra in eighth grade students for each readiness level can be found in Figures 1-7 


for significant interactions. 


Table B-6. Coefficients, standard errors, and significance for each outcome (CPS). 


Transition from 8'' to 9" grade 


High School Achievement & Graduation 


Math GPA | Math GPA | Absences Repeated Took Completed | Graduated 
in 8" Grade | in 9"" Grade | in 8" grade | algebra algebra II | core math __| in 4 years 
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before 11" 
grade 


Test Readiness 
Overall Effect p<0.0001 | p<0.0001 | p=0.1738 | p=0.9971 | p=0.7877 | p<0.0001 | p< 0.0001 
Algebra in 8th -0.19 0.16 -0.17 0.0% 0.3% 27.3% 2.5% 
(0.03) (0.02) (0.13) (0.002) (0.01) (0.01) (0.005) 
Algebra x Test Readiness 
Algebra & Stanine 1 0.33 -0.40 -1.17 12.9% 9.0% 0.9% 38.6% 
(0.16) (0.33) (0.91) (0.096) (0.12) (0.01) (0.08) 
Algebra & Stanine 2 0.14 -0.15 -0.91 -4.9% -4.4% -0.1% -8.0% 
(0.11) (0.15) (1.30) (0.007) (0.06) (0.01) (0.07) 
Algebra & Stanine 3 0.17 0.21 -1.31 -0.8% -4.1% 0.4% 0.1% 
(0.09) (0.12) (0.54) (0.015) (0.04) (0.01) (0.04) 
Algebra & Stanine 4 -0.04 0.18 -0.25 3.6% 1.5% 2.4% 6.4% 
(0.06) (0.08) (0.36) (0.014) (0.03) (0.01) (0.02) 
Algebra & Stanine 5 -0.13 0.31 -0.59 0.0% 0.0% 8.1% 4.5% 
(0.04) (0.04) (0.21) (0.007) (0.02) (0.01) (0.01) 
Algebra & Stanine 6 -0.24 0.19 -0.23 0.4% 0.5% 15.5% 2.0% 
(0.03) (0.04) (0.17) (0.004) (0.02) (0.01) (0.01) 
Algebra & Stanine 7 -0.27 0.14 -0.15 -0.1% 1.8% 25.3% 2.4% 
(0.03) (0.03) (0.16) (0.003) (0.02) (0.01) (0.01) 
Algebra & Stanine 8 -0.19 0.08 0.09 -0.8% -1.0% 41.6% 1.0% 
(0.04) (0.04) (0.18) (0.004) (0.02) (0.02) (0.01) 
Algebra & Stanine 9 -0.11 0.05 0.43 0.5% -2.5% 65.2% 0.5% 
(0.04) (0.06) (0.27) (0.003) (0.01) (0.02) (0.01) 
Me: Sa : fae p<0.0001 | p=0.014 | p=0.0411 | p<0.0001 | p=0.2498 | p<0.0001 | p<0.0001 
Algebra x GPA Readiness 
Algebra & F 0.04 0.15 -1.81 7.1% -8.2% 4.0% 3.3% 
(0.15) (0.11) (1.23) (0.05) (0.05) (0.02) (0.06) 
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Algebra&D | 0.10 0.09 0.16 5.1% 0.1% 2.7% 5.3% 
(0.08) (0.08) (0.46) (0.02) (0.03) (0.01) (0.03) 
Alcoa) 0.16 0.19 “0.36 0.6% 4.3% 11.3% 5.4% 
(0.04) (0.05) (0.19) (0.01) (0.02) (0.01) (0.01) 
Algebra &B| -0.24 0.19 0.21 0.5% 0.6% 21.0% 2.5% 
(0.03) (0.03) (0.15) (0.003) (0.02) (0.01) (0.01) 
Algebra& Al -0.20 0.10 0.15 0.5% 2.1% 42.2% 0.3% 
(0.03) (0.03) (0.16) (0.002) (0.01) (0.01) (0.01) 
HO: All algebrax GPA) |, _ 9 o418 | p= 0.1663 | p= 0.0283 | p=0.0267 | p= 0.0029 | p<0.0001 | p=0.0005 


Readiness Equal 
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